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In  many  cognitive  tasks,  such  as  problem  solving  (Simon,  1975),  solving 
analogies  (R.  Sternberg,  1977),  and  answering  questions  (Reder,  1982), 
individuals  appear  to  employ  a  variety  of  different  strategies.  Nowhere  is 
this  variability  in  strategy  use  so  apparent  as  in  decision-making.  A  major 
finding  of  the  last  decade  of  decision  research  is  that  an  individual  may  use 
many  different  kinds  of  strategies  in  making  a  decision,  contingent  upon  task 
demands  (Payne,  1982). 

Given  the  evidence  for  the  use  of  multiple  strategies  in  such  a  diverse 
set  of  cognitive  tasks,  a  fundamental  issue  is  how  people  decide  what  to  do. 
This  concern  with  how  problem  solvers  and  decision-makers  select  a  strategy  is 
reflected  in  the  growing  concernw  ith  the  regulation  of  cognition  and 
"metacognition"  (Brown,  Bransford,  Ferrara,  &  Camp ion i,  1983). 

An  approach  advocated  by  many  researchers  is  to  look  at  various 
strategies  as  having  differing  advantages  and  disadvantages,  and  hypothesize 
that  an  individual  might  select  the  strategy  that  is,  in  some  sense,  best  for 
the  task.  For  example,  Slegler  notes  that  "Children  (and  adults)  have  good 
reasons  to  use  multiple  strategies.  Strategies  differ  in  their  accuracy,  in 
how  long  they  take  to  execute,  in  their  demands  on  processing  resources,  and 
in  the  range  of  problems  to  which  they  apply,  (p.  1)"  Theorists  studying 
decision  behavior  have  expanded  upon  such  ideas  and  have  explicitly  viewed 
selection  among  decision  strateies  as  a  tradeoff  between  (1)  the  amount  of 
cognitive  resources  (mental  effort)  required  to  use  each  strategy,  and  (2)  the 
ability  of  each  strategy  to  produce  an  "accurate"  response  (Beach  and 
Mitchell,  1978;  Johnson  and  Payne,  1985;  Russo  and  Dosher,  1983;  Wright, 

1975). 

The  general  notion  that  different  processing  strategies  require 
different  amounts  of  computational  effort  to  execute  seems  obvious  in  the 


Strategy  Effort 
4 

domain  of  decision  making.  The  decision  strategy  of  expected  utility 
maximization,  for  instance,  requires  a  person  to  process  all  relevant  problem 
information  and  to  trade  off  values  and  beliefs.  The  lexicographic  choice 
rule  (Tversky,  1969),  on  the  other  hand,  chooses  the  alternative  which  is  best 
on  the  most  important  attribute,  ignoring  much  of  the  potentially  relevant 
problem  information.  Thus,  there  appear  to  be  clear  differences  among 
decision  strategies  in  the  amount  of  information  that  is  processed  in  making  a 
choice. 

At  a  more  precise  level  of  analysis,  however,  a  comparison  among 
decision  strategies  in  terms  of  mental  effort  is  much  more  difficult.  In  part 
this  is  because  the  decision  strategies  that  have  been  proposed  in  the 
literature  have  varied  widely  in  terms  of  their  formal  expression.  Some  have 
been  proposed  as  formal  mathematical  models  (e.g.,  elimination-by-aspects, 
Tveraky ,  1972),  and  others  as  verbal  process  descriptions  (e.g.,  the  majority 
of  confirming  dimensions  rule,  Russo  &  Dosher, 1983) .  What  is  needed  is  a 
language  that  could  be  used  to  express  a  diverse  set  of  decision  strategies  in 
terms  of  a  common  set  of  cognitive  operations.  Such  a  language  would  also 
allow  a  more  detailed  analysis  of  the  components  of  processing  (effort) 
involved  when  a  particular  decision  strategy  is  used  to  solve  a  particular 
decision  problem.  In  other  words,  one  could  examine  whether  the  amount  of 
information  to  be  processed  is  the  major  determinant  of  effort,  or  whether  the 
specific  mix  of  cognitive  operations  which  is  utilized  affects  effort. 

Another  difficulty  facing  such  cost-benefit  approaches,  in  addition  to 
conceptualizing  effort,  is  actually  measuring  the  effort  associated  with  a 
given  strategy.  There  have  been  a  number  of  measurement  techniques  proposed 
for  the  related  concept  of  mental  workload  (Gopher  &  Donchin,  in  press; 
Wlckens,  1984),  ranging  from  self-reports  to  physiological  measures.  However, 
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the  different  measures  of  workload,  such  as  response  latencies,  secondary 
tasks,  error  rates,  or  self-reports,  do  not  always  agree.  Hence,  Gopher  and 
Donchin  (in  press)  recommend  the  use  of  a  battery  of  multiple  measures,  along 
with  a  detailed  theoretical  analysis  of  the  expected  workload  of  a  task. 

The  primary  purpose  of  this  paper  is  to  examine  the  effort  required  to 
use  various  decision  strategies  in  choice  environments  that  vary  on  several 
dimensions.  We  first  develop  a  metric  of  decision  effort  based  on  the  concept 
of  elementary  information  processes  (Chase,  1978;  Newell  &  Simon,  1972).  We 
then  use  this  component ial  approach  to  modeling  decision  effort  to  predict 
multiple  indicators  of  strategy  execution  efforts  decision  latencies,  self- 
reports  of  task  difficulty,  and  errors  in  strategy  execution.  The  independent 
variables  used  include  measures  based  upon  the  proposed  componential  approach 
to  decision  effort  as  well  as  some  alternative  models.  Our  goal  is  not  to 
propose  a  complete  theory  of  mental  workload,  but  to  illustrate  an  approach  to 
measuring  the  execution  effort  of  choice  strategies.  Such  an  approach  may 
allow  us  to  better  understand  when  a  particular  decision  strategy  will  be  used 
to  solve  certain  decision  problems. 

A  secondary  goal  of  the  current  research  is  to  provide  evidence  that 
decomposition  in  general  is  a  useful  concept  in  decision-making.  While 
commonplace  throughout  much  of  cognitive  psychology,  the  notion  of  dividing 
strategies  into  a  small  set  of  shared  components  is  relatively  untested  in 
decision-making. 

In  the  following  section,  previous  attempts  to  conceptualize  and  measure 
decision  effort  are  briefly  addressed,  and  the  proposed  approach  is  outlined. 
Then  the  methodology  and  results  of  a  study  designed  to  test  this  approach  are 


described  in  detail. 
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Measuring  Decision  Effort 

The  theoretical  construct  of  mental  effort  has  a  long  and  venerable 
history  in  psychology  (Kahneman,  1973;  Navon  &  Gopher,  1979;  Thomas,  1983). 
However,  there  have  been  only  a  few  attempts  to  model  and  compare  decision 
rules  in  terms  of  an  effort  metric. 

Two  studies  that  attempted  to  directly  measure  the  execution  effort  of 
various  decision  rules  are  Wright  (1975)  and  Bettman  &  Zins  (1979).  In  each 
study,  subjects  were  instructed  to  use  particular  decision  rules  to  solve 
certain  problems.  The  percent  of  correct  judgments  using  the  rules  and  self- 
reports  of  task  difficulty  or  ease  of  use  were  obtained  in  both  studies.  In 
addition,  Bettman  and  Zins  obtained  a  measure  of  the  time  taken  to  apply  a 
rule  to  a  problem.  The  results  clearly  show  that  certain  rules  were  perceived 
as  less  effortful  then  others.  For  example ,  a  lexicographic  rule  was 
generally  perceived  as  less  effortful  than  other  decision  rules.  That  rule 
also  tended  to  be  the  most  accurate  and  quickest  in  its  execution.  However, 
these  two  studies  had  significant  limitations.  First,  neither  study  employed 
a  method  beyond  initial  instruction  to  ensure  that  subjects  actually  used  the 
prescribed  decision  rules.  Second,  neither  study  provided  a  conceptual  basis 
(model)  for  why  a  certain  decision  rule  would  be  expected  to  be  more  or  less 
effortful  in  a  particular  task.  That  is,  neither  study  attempted  to  model  the 
components  of  decision-making  effort. 

Shugan  (1980)  suggested  an  effort  metric  based  upon  one  operation,  the 
binary  comparison  of  two  alternatives  on  an  attribute.  More  effortful 
decisions  involved  more  comparisons.  Shugan  also  showed  that  the  effort  of 


strategies  would  vary  with  certain  task  characteristics  like  the  correlational 
structure  among  attributes.  Unfortunately,  using  the  binary  comparison  as  the 
fundamental  unit  of  effort  restricts  Shugan' s  analysis  to  certain  decision 
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rules.  However,  Shugan's  work  implies  that  any  approach  to  modeling  strategy 
effort  must  be  sensitive  to  the  joint  effects  of  strategy  and  task. 

Based  upon  the  work  of  Newell  and  Simon  (1972),  Huber  (1980)  and  Johnson 
(1979)  offered  decompositions  of  choice  strategies  using  more  extensive  sets 
of  components.  Each  independently  suggested  that  decision  strategies  be 
described  by  a  set  of  elementary  information  processes  (EIP's).  A  decision 
rule  or  strategy  was  represented  as  a  sequence  of  mental  events,  such  as 
reading  a  piece  of  information  into  STM  (short-term  memory),  multiplying  a 
probability  and  a  payoff*,  or  comparing  the  values  of  two  alternatives  on  an 
attribute.  Johnson  and  Payne  (1985)  employed  a  similar  set  of  EIP's  for 
decision  making,  shown  in  Table  1,  and  constructed  production  system 
implementations  of  several  different  choice  strategies. 

We  propose  this  set  of  EIP's  as  underlying  components  from  which  various 
decision  strategies  can  be  constructed.  That  is,  we  assume  that  a  set  of 
EIP's  like  those  in  Table  1  is  at  a  sufficient  level  of  detail  to  provide  a 
common  language  to  describe  the  diverse  set  of  decision  strategies  that  exist. 
Furthermore,  we  propose  that  by  using  such  EIP's  to  describe  strategies  at  the 
componential  level,  meaningful  comparisons  among  strategies  in  terms  of 
decision  effort  can  be  made.2 

Insert  Table  1  about  here 

Given  a  decomposition  of  decision  strategies  into  a  common  set  of 
components,  a  general  measure  of  decision  effort  is  the  number  of  component 
EIP's  required  to  execute  a  particular  strategy  in  a  particular  task 
environment.  This  notion  of  measuring  decision  effort  in  terms  of  the  number 
of  EIP's  builds  on  an  idea  for  measuring  processing  effort  proposed  by  Newell 
and  Simon  (1972).  Empirical  support  for  this  approach  has  been  provided  by 
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showing  a  relationship  between  the  predicted  number  of  EIP's  used  and  response 
times  for  a  variety  of  cognitive  tasks  (Card,  Moran,  &  Newell,  1983;  Carpenter 
&  Just,  1975). 

To  validate  this  particular  proposed  decomposition  into  EIP's,  we 
examine  several  alternative  models.  Specifically,  we  investigate  several 
models  of  decision  effort  in  a  setting  where  subjects  make  choices  for  six 
sets  of  twenty  decision  problems  varying  in  size  and  other  factors,  using  a 
different  decision  strategy  for  each  set.  The  simplest  model  of  decision 
effort  in  terms  of  EIP's  would  be  to  treat  each  component  process  as  equally 
effortful  and  simply  sum  the  numbers  of  each  component  process  to  get  an 
overall  measure  of  effort.  Alternatively,  the  effort  required  by  individual 
components  could  be  estimated  and  the  tally  for  the  individual  operations 
weighted  by  those  estimates  before  summing  across  all  components.  In  this 
study  we  will  examine  both  of  these  versions  of  the  componential  approach  to 
measuring  decision  effort  in  terms  of  EIP's. 

Alternative  models  of  effort  not  based  on  the  componential  approach  are 
also  considered.  At  the  structural  level,  we  characterize  effort  as  a 
function  of  task  complexity  (number  of  alternatives  and  attributes)  and  the 
particular  strategy  used.  Next,  we  investigate  a  model  of  effort  based  on  the 
number  of  items  of  information  processed  by  a  particular  strategy  in  a  choice 
environment.  Since  it  is  easy  to  monitor  information  acquisition  behavior, 
this  might  be  called  an  explicit  behavioral  model  of  strategy  effort.  Third, 
we  will  examine  EIP  models  of  decision  effort  based  upon  the  conceptualization 
of  the  underlying  components  of  decision  processes  outlined  above.  Finally, 
we  explore  models  of  decision  effort  that  combine  the  EIP  concept  with 
additional  factors,  such  as  the  task  complexity  variables  and  individual 
differences  variables. 
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E«ch  model  of  strategy  effort  will  be  investigated  using  three 
indicators  of  execution  effort:  (1)  the  time  to  make  a  response,  (2)  self- 
reports  of  effort,  and  (3)  the  number  of  errors  in  the  execution  of  a 
strategy.  For  the  models  using  response  times  as  the  measure  of  effort, 
analyses  can  be  carried  out  at  different  levels  of  aggregation:  the  overall 
decision,  acquisitions  of  individual  items  of  information,  and  intermediate 
levels  of  aggregation  (e.g.,  total  time  spent  on  each  individual  alternative 
(Card,  Moran,  &  Newell,  1983)).  The  overall  goal  of  performing  these  multiple 
analyses  using  different  models  of  effort,  different  indicators  of  effort,  and 
different  levels  of  aggregation  is  to  attempt  to  determine  the  most 
appropriate  conceptualization  for  measuring  the  effort  required  to  execute 
decision  strategies.  We  hypothesize  that  the  componential  approach  described 
above  will  provide  the  best  fits  to  the  data  across  the  varying  levels  of 
analysis.  If  the  componential  approach  can  be  shown  to  be  both  robust  over 
different  indicators  of  effort  and  different  levels  of  aggregation,  and  at  the 
same  time  superior  to  alternative  models,  we  will  have  provided  strong  support 
for  this  approach  to  conceptualizing  decision  making  effort. 

Method 

Overview 

Subjects  were  trained  to  use  six  different  decision  strategies  for 
making  these  decisions.  Each  strategy  was  used  in  a  separate  session  to  make 
twenty  decisions  for  decision  problems  ranging  in  size  from  two  to  six 
alternatives  and  from  two  to  four  attributes.  Subjects  used  a  computer-based 
Information  acquisition  system  to  acquire  information  and  make  decisions  among 
sets  of  alternatives.  The  computer-based  acquisition  system  monitored  the 
subjects'  information  sequences;  recorded  latencies  for  each  acquisition; 
recorded  the  overall  time  for  each  problem;  and  recorded  any  errors  made  by 
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the  subject  (i.e.,  departures  from  the  prescribed  search  pattern  or  choice). 

In  addition,  subjects  rated  the  difficulty  of  each  choice  and  the  effort  each 
required  on  two  response  scales  presented  at  the  end  of  each  decision  problem. 

The  data  recorded  by  the  computer-based  acquisition  system  provided  for 
three  major  types  of  dependent  variable:  response  times,  self-reports  of 
effort  and  difficulty,  and  observations  of  errors  in  execution  of  a  strategy. 
Three  basic  models  were  used  to  generate  independent  variables:  structural, 
behavioral,  and  EIP,  as  noted  above.  These  models  were  then  used  to  predict 
the  three  types  of  dependent  measures.  Regression  models  were  used  for  the 
response  time  and  self-report  data,  and  logistic  regressions  were  used  to 
predict  the  error  data.  Finally,  for  the  response  time  data,  the  EIP  model 
analyses  were  performed  at  three  levels  of  aggregation:  the  overall  time  for 
each  problem,  the  time  for  each  individual  acquisition,  and  the  aggregate 
times  spent  on  each  alternative  and  on  each  attribute  in  a  problem  (e.g., 
seven  separate  times  for  a  four  alternative,  three  attribute  problem). 

He  describe  the  details  of  the  methodology  as  follows:  first,  the  six 
decision  strategies  used  are  described,  followed  by  a  description  and  examples 
of  b~v  the  EIP  counts  were  generated.  Then  the  generation  of  the  sets  of 
twenty  decision  problems  is  discussed,  followrd  by  details  on  the  computer- 
based  acquisition  system.  Finally,  the  experimental  procedure  is  discussed  in 
detail,  preliminary  analyses  are  reported,  and  an  overview  of  the  major 
analyses  performed  is  presented. 

Decision  Strategies 

Rules  Used.  Six  different  decision  strategies  were  used  in  the 


experiment:  weighted  additive;  equal  weighted  additive;  lexicographic; 
elimination  by  aspects;  satisficing  (conjunctive);  and  majority  of  confirming 
dimensions.  Each  of  these  rules  was  implemented  as  a  production  system  model 
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(for  examples  see  Johnson  and  Payne  1985;  Payne,  Bettman,  and  Johnson,  1986). 
These  particular  rules  were  selected  for  two  reasons:  1)  each  rule  has  been  a 
focus  of  previous  research  on  choice  processes;  and  2)  this  set  of  rules 
provides  a  broad  coverage  of  the  set  of  basic  elementary  operations  (EIP's) 
used  as  the  components  in  our  conceptualization  of  strategy  execution  effort. 
We  first  describe  the  strategies,  and  then  the  elementary  operations  are 
considered. 

To  facilitate  our  description,  we  first  outline  a  typical  choice 
problem.  A  choice  problem  consists  of  a  set  of  alternatives,  each  of  which  is 
described  on  several  attributes,  or  criteria.  In  this  study,  the  alternatives 
were  job  candidates,  and  the  attributes  were  scores,  or  ratings,  on  various 
selection  criteria  (e.g.,  leadership  potential  and  motivation).  The  decision 
problems  had  from  two  to  six  candidates,  each  described  by  from  two  to  four 
criteria.  For  each  attribute,  an  importance  weight  and  a  cutoff  value 
specifying  a  minimally  acceptable  level  for  that  attribute  were  also 
displayed.  Different  decision  strategies  might  use  both  weights  and  cutoffs, 
one  of  the  two,  or  neither,  as  described  below. 

The  weighted  additive  rule  requires  the  subject  to  develop  an  evaluation 
for  each  alternative  by  multiplying  each  weight  times  the  attribute  rating  and 
adding  those  products  for  all  attributes.  The  alternative  with  the  highest 
evaluation  is  selected.  In  the  equal  weighted  additive  model,  the  evaluation 
for  each  alternative  is  obtained  by  adding  the  ratings  for  all  the  attributes, 
with  the  alternative  with  the  highest  evaluation  selected.  No  weights  or 
cutoffs  are  used. 

The  lexicographic  rule  requires  the  subject  to  first  find  the  most 
important  attribute  (the  attribute  with  the  largest  weight)  and  then  search 
the  values  on  that  attribute  for  the  alternative  with  the  highest  value.  That 
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alternative  is  selected,  unless  there  are  ties.  In  this  case,  those  tied 
alternatives  are  examined  on  the  second  most  important  attribute.  That 
process  continues  until  a  winner  is  found. 

The  elimination  by  aspects  (EBA)  strategy  also  begins  by  determining  the 
most  important  attribute  and  examining  that  attribute's  cutoff  value.  Next, 
all  alternatives  with  ratings  below  the  cutoff  for  that  attribute  are 
eliminated.  This  process  continues  with  the  second  most  important  attribute, 
and  so  on,  until  one  alternative  remains.  The  satisficing  (conjunctive)  rule 
requires  the  subject  to  consider  one  alternative  at  a  time,  comparing  each 
attribute  to  the  cutoff  value.  If  any  attribute  is  below  the  cutoff  value, 
that  alternative  is  rejected.  The  first  alternative  which  has  values  which 
pass  the  cutoff  for  all  attributes  is  chosen. 

Finally,  the  majority  of  confirming  dimensions  rule  (HCD)  processes 
pairs  of  alternatives.  The  values  of  the  two  alternatives  are  compared  for 
each  attribute,  and  a  running  evaluation  ia  kept:  if  the  first  alternative 
has  a  greater  value  on  an  attribute  than  the  second,  one  is  added  to  the 
score;  if  the  second  alternative  is  greater,  one  is  subtracted;  if  the  two 
alternatives  are  tied,  the  score  is  not  changed.  After  all  attributes  have 
been  examined,  if  the  score  is  positive,  the  first  alternative  is  retained;  if 
the  score  is  negative,  the  second  alternative  is  retained;  and  if  the  score  is 
zero,  the  alternative  winning  the  comparison  on  the  last  attribute  is 
retained.  Thus,  the  general  idea  is  to  retain  the  alternative  which  is  better 
on  the  most  criteria.  The  alternative  which  is  retained  is  then  compared  to 
the  next  alternative  remaining  among  the  set  of  alternatives.  If  no  other 
alternative  remains,  the  retained  alternative  is  selected. 
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Calculating  EIP  Counts 

To  describe  the  steps  a  subject  followed  in  more  detail  and  to  show  how 
EIP  counts  were  determined,  we  first  consider  the  particular  EIP's  used  and 
then  present  two  more  detailed  examples  of  rules  applied  to  a  particular 
decision  problem.  The  major  EIP's  utilized  were  MOVES,  READS,  ADDITIONS, 
PRODUCTS,  COMPARES,  ELIMINATIONS,  and  DIFFERENCES.  A  MOVE  involves  moving  to 
another  piece  of  information,  while  a  READ  consists  of  acquiring  that 
information  (moving  it  to  short  term  memory).  Since  MOVES  and  READS  are  in 
general  perfectly  correlated,  we  will  only  consider  READS  (acquisitions)  in 
this  study.  ADDITIONS,  PRODUCTS  (of  weights  and  ratings),  and  DIFFERENCES  are 
self “evident.  COMPARES  involved  comparing  two  pieces  of  information  and 
determining  the  larger  (two  ratings,  two  overall  alternative  scores,  two 
weights,  a  rating  and  cutoff,  etc.).  Finally,  ELIMINATIONS  could  be  either 
discarding  an  attribute  (because  it  had  already  been  used)  or  an  alternative 
(because  its  score  was  surpassed,  it  failed  a  cutoff,  etc.). 

Examples.  Two  examples  will  be  considered  in  more  detail,  a  weighted 
adding  case  and  an  EBA  example.  Before  doing  this,  however,  some  general 
comments  are  in  order.  First,  the  number  of  EIP's  required  for  a  particular 
decision  is  a  function  of  the  specific  rule  used,  the  size  of  the  problem  (the 
number  of  alternatives  and  attributes),  and  the  specific  values  of  the  data. 
Rules  which  examine  all  of  the  ratings  for  each  alternative,  such  as  the 
weighted  adding  rule,  need  more  EIP's  than  rules  which  may  process  only  part 
of  the  data,  such  as  the  EBA  rule.  Larger  problems  also  tend  to  require  more 
EIP's.  Problems  with  more  values  which  surpass  cutoffs  will  also  generally 
require  more  EIP's.  Second,  in  the  specification  of  the  rules,  an  attempt  was 
made  to  take  advantage  of  the  left  to  right,  top  to  bottom  natural  reading 


order. 


Insert  Table  2  about  here 
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For  the  weighted  adding  rule,  consider  the  4  alternative,  3  attribute 
decision  problem  shown  in  Table  2.  The  numbers  in  parentheses  are  labels  that 
will  be  used  for  convenience  for  identifying  the  sequence  of  acquisitions  in 
the  following.  Subjects  were  instructed  to  acquire  the  first  weight  (1)  and 
then  the  rating  on  the  first  attribute  (4).  They  then  multiplied  these  two 
numbers  and  retained  the  score.  This  process  was  repeated  (sequence  (2),  (5), 
(3),  (6))  until  alternative  A  was  finished.  For  the  first  alternative,  the 
total  score  of  60  was  simply  retained  as  the  current  best.  After  processing 
the  first  alternative,  there  would  be  six  READS,  three  PRODUCTS,  two  ADDS,  and 
no  COMPARISONS,  DIFFERENCES,  or  ELIMINATIONS.  For  alternative  B,  the  sequence 
would  be  (1),  (7),  (2),  (8),  (3),  (9).  Then  the  total  score  for  B,  44,  would 
be  compared  to  the  current  best,  and  the  current  best  of  60  would  be  retained. 
The  assumption  was  made  that  in  the  comparison  of  total  scores,  the  losing 
alternative  was  not  explicitly  eliminated.  Rather,  the  subject  would  merely 
store  the  one  retained.  Thus,  after  two  alternatives  we  would  have  twelve 
READS,  six  PRODUCTS,  four  ADDS,  one  COMPARISON,  no  DIFFERENCES,  and  no 
ELIMINATIONS.  This  process  would  be  repeated  for  the  remaining  two 
alternatives  (sequence  (1),  (10),  (2),  (11),  (3),  (12),  (1),  (13),  (2),  (14), 
(3),  (15)).  Hence,  the  production  system  model  predicts  that  in  total  this 
problem  would  require  24  READS,  8  ADDITIONS,  12  PRODUCTS,  3  COMPARISONS ,  no 
DIFFERENCES,  and  no  ELIMINATIONS. 

Insert  Table  3  about  here 

The  example  of  a  three  alternative,  four  attribute  problem  shown  in 
Table  3  is  used  to  clarify  the  EBA  rule  specification.  The  subject  had  to 
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first  find  the  most  important  attribute.  This  was  done  by  starting  with  the 
first  weight  and  comparing  it  to  the  second,  retaining  the  larger  (the 
second).  The  second  was  then  compared  to  the  third,  and  the  second  was 
retained.  Then  the  second  was  compared  to  the  fourth,  and  the  fourth 
(experience)  was  retained  as  the  most  important  attribute.  The  sequence  of 
acquisitions  would  thus  be  (1),  (2),  (3),  (4).  There  would  be  four  READS  and 
three  COMPARISONS.  Then  the  subject  acquired  the  cutoff  for  experience  and 
examined  the  value  for  all  alternatives  on  experience,  comparing  each  value  to 
the  cutoff  and  eliminating  any  alternative  not  passing  the  cutoff.  In  this 
case,  the  sequence  would  be  (8),  (12),  (16),  and  (20),  with  alternative  C 
eliminated.  The  total  EIP's  thus  far  would  be  eight  READS,  six  COMPARISONS, 
and  one  ELIMINATION.  Then  the  experience  attribute  would  be  eliminated,  and 
the  weights  for  the  remaining  three  criteria  would  be  acquired  and  compared, 
resulting  in  motivation's  being  selected  as  the  second  most  important 
attribute  (sequence  (1),  (2),  (3)).  Then  the  cutoff  for  motivation  was 
acquired  and  the  values  for  the  retained  alternatives,  A  and  B,  were  compared 
to  the  cutoff  (sequence  (6),  (10),  (14)).  At  this  point,  there  would  be  a 
total  of  14  READS,  10  COMPARISONS,  and  two  ELIMINATIONS.  Both  A  and  B  passed 
the  cutoff,  so  the  subject  would  then  eliminate  the  motivation  attribute  and 
return  to  the  weights  to  determine  the  third  most  important  remaining 
attribute,  leadership  (sequence  (1),  (3)).  Then  the  cutoff  for  leadership  was 
examined,  A  and  B  were  compared  to  the  cutoff,  and  A  was  eliminated.  B  would 
then  be  chosen  (sequence  (5),  (9),  (13).  In  total,  there  would  be  19  READS, 

13  COMPARISONS,  and  four  ELIMINATIONS  (two  attributes  and  two  alternatives). 

These  examples  illustrate  two  principles!  the  number  of  EIP's  varies 
with  problem  size  and  with  the  particular  values  used,  and  different  rules  use 
different  subsets  of  the  EIP's.  Vith  regard  to  the  second  point,  the  weighted 
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adding  rule  uses  READS,  ADDITIONS,  MULTIPLICATIONS,  and  COMPARISONS ;  the  equal 
weighted  adding  rule  uses  READS,  ADDITIONS,  and  COMPARISONS;  the  lexicographic 
rule  uses  READS,  COMPARISONS,  and  ELIMINATIONS;  the  EBA  rule  uses  READS, 
COMPARISONS ,  and  ELIMINATIONS;  the  satisficing  rule  uses  READS,  COMPARISONS , 
and  ELIMINATIONS;  and  the  MCD  rule  uses  READS,  ADDITIONS,  COMPARISONS , 
ELIMINATIONS,  and  DIFFERENCES. 

It  should  also  be  noted  that  certain  rules  (weighted  adding,  equal 
weighted  adding)  have  the  same  EIP  counts  for  any  problems  of  the  same  size 
(i.e.,  with  the  same  number  of  alternatives  and  attributes).  On  the  other 
hand,  the  other  rules  (lexicographic,  EBA,  satisficing,  and  MCD)  can  have 
different  EIP  counts  even  for  problems  of  the  same  size,  depending  upon  the 
particular  values  of  the  data.  This  property  of  the  rules  affected  the 
selection  of  decision  problems  for  the  experiment,  as  discussed  next. 

Selection  of  the  Decision  Problems 

As  noted  above,  subjects  completed  twenty  decision  problems  for  each  of 
the  six  decision  rules.  These  decision  problems  were  generated  by  taking 
several  factors  into  account.  First,  pilot  studies  revealed  that  numbers  of 
attributes  greater  than  four  were  extremely  difficult  for  subjects, 
particularly  for  the  weighted  adding  rule.  Second,  numbers  of  alternatives 
greater  than  six  caused  crowding  problems  on  the  computer  display  used  in  the 
Information  acquisition  system.  Hence,  decision  problems  with  from  two  to  six 
alternatives  and  two  to  four  attributes  were  used.  This  generated  15  possible 
sizes,  ranging  from  two  alternatives  and  two  attributes  to  six  alternatives 
and  four  attributes. 

For  the  weighted  adding  and  equal-weighted  adding  rules,  since  problem 
size  determines  the  EIP  count,  one  problem  of  each  size  was  included,  making 
fifteen  decision  problems.  Then  five  problem  sizes  were  randomly  selected  to 
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complete  the  twenty  decision  problems.  Values  for  the  weights  and  ratings 
were  assigned  randomly,  with  the  restriction  that  no  overall  scores  for 
alternatives  in  the  same  problem  set  were  tied. 

For  the  remaining  rules,  several  problems  were  generated  for  each 
problem  size  that  represented  low,  intermediate  and  high  EIP  counts  for  that 
size  Ce.g.,  for  a  three  alternative,  four  attribute  EBA  problem,  elimination 
of  two  alternatives  on  the  first  attribute  would  lead  to  a  low  count, 
retention  of  all  three  alternatives  until  the  last  attribute  would  be  a  high 
count,  and  the  operations  used  for  the  example  described  above  might  be  an 
intermediate  count).  Then  sets  of  twenty  problems  were  randomly  selected  for 
each  rule  from  the  total  set  of  forty-five  size/count  combinations. 

The  random  selection  procedure  just  described  was  repeated  many  times  in 
an  attempt  to  deal  with  correlation  problems  among  the  EIP  counts.  Since  the 
EIP  counts  were  to  be  used  as  independent  variables  in  models  to  predict 
decision  times,  effort  self-reports,  and  errors,  it  was  desirable  that  their 
intercorrelations  across  all  120  decision  problems  should  be  as  low  as 
possible  to  avoid  multi-collinearity  problems  (Kmenta  1971).  As  noted  above, 
however,  certain  rules  use  only  some  EIP's  and  not  others,  so  there  are  some 
correlations  that  will  be  high  because  of  the  definition  of  the  rules.  For 
example,  the  correlation  between  COMPARISONS  and  ELIMINATIONS  will  tend  to  be 
high  because  rules  with  no  ELIMINATIONS  (e.g.,  the  adding  rules)  tend  to  do 
very  few  COMPARISONS,  whereas  rules  with  many  COMPARISONS  also  have  more 
ELIMINATIONS.  To  minimize  these  intercorrelation  problems,  we  repeated  the 
random  selection  procedure  1,000  times,  and  selected  the  set  of  120  decision 
problems  with  the  smallest  intercorrelations.  The  resulting  intercorrelations 
are  shown  in  Table  4.  We  were  unable  to  further  reduce  the  highest,  COMPARES 
and  ELIMINATIONS,  for  the  reasons  outlined  above.  Although  these 
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intercorrelations  will  present  some  interpretation  problems  for  the  analysis 
of  the  overall  decision  times,  the  EIP's  at  other  levels  of  aggregation  have 
lower  intercorrelations. 

Insert  Table  4  about  here 

The  Computer-Based  Information  Acquisition  System 

A  computer-based  information  acquisition  system  was  utilized  in  carrying 
out  the  experiment.  A  software  system  for  personal  computers,  Mouselab 
(Johnson,  Payne,  Schkade,  &  Bettman,  1986),  was  developed  to  monitor 
information  acquisition  patterns  and  decisions  of  subjects.  Mouselab  can 
present  several  different  types  of  information  displays  to  subjects.  In  this 
study,  the  subject  saw  a  matrix  display  on  the  computer  monitor  for  each 
decision  problem.  The  rows  of  the  matrix  were  labeled  weights,  cutoffs,  and 
then  the  names  of  the  alternatives  to  be  considered.  The  columns  were  labeled 
with  the  names  of  the  attributes.  At  the  bottom  of  the  monitor  screen  were 
boxes  used  to  Indicate  choice  of  an  alternative  (hence  termed  choice  boxes). 
For  an  example  of  this  display,  see  Figure  1. 

Insert  Figure  1  about  here 

Initially,  the  matrix  display  provides  only  the  labels  for  the  rows  and 
columns  and  the  choice  boxes.  The  information  is  hidden  in  the  blank  cells  on 
the  screen.  To  acquire  information,  the  subject  must  move  a  cursor  controlled 
by  the  mouse  to  the  desired  cell  of  the  matrix.  The  cell  then  opens, 
displaying  the  information.  For  each  decision,  the  subject  would  use  the 
mouse  to  acquire  the  appropriate  information  in  the  sequence  specified  by  the 
current  strategy.  Mouselab  recorded  the  sequence  in  which  cells  were  opened 
and  the  time  spent  in  each  cell.  The  time  measurements  use  the  system  clock 
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of  the  personal  computer,  providing  a  resolution  of  approximately  17 
milliseconds.  After  the  requisite  information  had  been  examined,  the  subject 
moved  to  the  appropriate  choice  box  and  clicked  a  button  on  the  mouse  to 
designate  the  chosen  alternative.  Mouselab  can  also  present  response  scales 
and  text  Instructions  to  subjects,  as  noted  below. 

A  crucial  feature  of  Mouselab  for  the  present  study  is  the  ability  to 
monitor  the  sequence  of  acquisitions  made  by  a  subject.  Since  the  EIP  models 
of  effort  we  propose  required  EIP  counts  for  each  problem,  it  is  crucial  that 
subjects  use  the  strategy  exactly  as  it  is  specified,  so  that  the  EIP  counts 
can  be  predicted  accurately.  For  example,  to  ensure  that  the  EIP  counts  for 
the  weighted  adding  and  EBA  examples  given  above  are  correct,  we  must  monitor 
that  subjects  follow  the  exact  acquisition  sequence  for  each  rule.  Mouselab 
includes  a  move  monitoiing  feature,  which  allows  the  correct  sequence  of  cells 
to  be  specified  for  each  decision  problem.  If  the  subject  enters  a  "wrong" 
cell,  the  -ell  will  not  open,  and  after  two  seconds  the  computer  will  emit  an 
audible  buzz.  The  attempt  to  enter  an  incorrect  cell  is  also  recorded  in  the 
output  information  about  the  subject's  move  sequence.  Hence,  trials  where  a 
specified  number  of  incorrect  moves  has  occurred  can  later  be  discarded  or 
analyzed  as  error  trials  if  desired. 

A  mouse-based  information  acquisition  system  was  used  for  several 
reasons.  Card,  Moran,  and  Newell  (1983)  compared  the  mouse,  joystick,  and  two 
keyboard-based  devices.  They  found  that  the  mouse  was  easy  to  learn,  was 
significantly  faster  than  the  other  devices,  and  had  a  lower  error  rate.  An 
analysis  of  a  typical  decision  task  for  this  study  using  Fitts  Law  indicates 
that  subjects  could  move  between  information  cells  in  less  than  100 
milliseconds.  This  suggests  that  the  time  to  move  the  mouse  is  limited  mainly 
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by  the  time  it  takes  to  think  where  to  point,  not  the  movement  of  the  mouse 
itself. 

Procedure 

Overview.  Subjects  participated  in  eight  separate  sessions  over  a 
period  of  several  days.  Each  session  lasted  from  one  to  one  and  a  half  hours. 
No  more  than  two  sessions  were  run  in  one  day,  and  separate  sessions  were  at 
least  four  hours  apart.  The  first  session  taught  subjects  the  decision  rules 
and  familiarized  them  with  the  computer-based  information  acquisition  system. 
In  each  of  the  subsequent  six  estimation  sessions,  a  subject  made  twenty 
choices  using  a  different  specified  rule.  The  order  of  the  rules  was 
randomized  across  subjects.  The  final  session  had  twelve  choice  problems 
where  the  subject  was  free  to  use  any  strategy  desired.  These  "free"  choices 
are  not  analyzed  further  in  this  report. 

Subjects.  Subjects  were  seven  adults,  ranging  in  age  from  21  to  34,  and 
included  four  males  and  three  females.  They  varied  in  their  prior  awareness 
of  the  decision  making  literature,  ranging  from  graduate  students  who  had 
studied  decision  making  to  non-students  who  had  never  been  exposed  to  those 
concepts. 

Training.  It  was  crucial  that  subjects  thoroughly  learn  the  six 
decision  strategies  to  be  used  (weighted  adding,  equal-weighted  adding, 
lexicographic,  elimination  by  aspects,  satisficing,  and  majority  of  confirming 
dimensions)  and  learn  to  use  the  mouse-based  acquisition  system.  Hence,  a 
familiarization  session  was  developed.  Subjects  were  first  introduced  to  the 
mouse  and  were  shown  how  to  use  it  to  open  the  cells,  respond  to  various 
response  scales,  and  indicate  a  choice.  After  practicing  these  tasks, 
subjects  were  next  given  a  training  session  for  the  decision  rules  which  was 
developed  using  the  Mouselab  system. 
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The  subject  was  first  informed  about  the  type  of  decision  problem  to  be 
presented.  He  or  she  was  informed  that  the  decisions  to  be  made  were 
personnel  decisions  involving  selection  of  job  candidates.  These  selections 
were  to  be  made  according  to  the  rules  specified  by  different  divisions  of 
their  company,  and  the  sets  of  candidates  might  have  both  differing  numbers  of 
candidates  and  different  amounts  of  information  on  each  candidate.  Subjects 
were  then  told  that  information  on  up  to  four  attributes  might  be  presented: 
leadership  potential,  creativity,  job  experience,  and  motivation.  The  left  to 
right  ordering  of  the  subset  of  these  attributes  used  on  any  given  trial  was 
randomized . 

Following  this  overview  of  the  problem  setting,  subjects  were  introduced 
to  the  ratings  used  to  describe  each  candidate  on  each  attribute.  Ratings 
ranging  from  2  (poor)  to  7  (excellent)  were  used  as  the  information  in  each 
cell.  Subjects  were  next  introduced  to  the  ideas  of  importance  weights  for 
the  attributes  and  cutoffs  for  the  attributes.  They  were  then  asked  to  select 
the  most  important  attribute  and  to  pick  candidates  surpassing  a  cutoff  to 
provide  training  using  these  ideas.  These  concepts  were  then  reviewed  before 
the  decision  rules  were  introduced. 

For  each  rule,  the  subject  was  first  given  a  thorough  written 
description  of  the  rule  on  the  computer  monitor.  Then  the  subject  was  given 
several  decision  problems  and  told  to  apply  the  rule  using  the  mouse.  The 
move  monitoring  system  was  used  on  the  last  trial  to  Inform  subjects  of 
mistakes.  The  subject  was  also  told  what  the  correct  choice  using  the  rule 
should  have  been.  Thus,  subjects  had  accuracy  feedback  on  both  the  sequence 
of  acquisitions  and  their  choices  during  training.  Following  these  practice 
trials,  the  next  rule  was  presented.  The  rules  were  presented  in  the 
familiarization  session  in  an  order  ranging  from  simple  to  more  complex: 
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equal-weighted  adding,  lexicographic,  satisficing,  elimination  by  aspects, 
weighted  adding,  and  majority  of  confirming  dimensions. 

Finally,  after  all  six  rules  had  been  presented,  subjects  were  given  six 
practice  trials,  one  for  each  rule.  These  trials  introduced  the  use  of  two 
response  scales  to  measure  the  difficulty  of  the  decision  task  and  how 
effortful  the  decision  was.  The  first  scale  asked  the  subject  to  rate  how 
difficult  the  choice  was  to  make  on  a  scale  ranging  from  0  (not  difficult  at 
all)  to  10  (extremely  difficult).  The  second  scale  asked  the  subject  to  rate 
how  much  effort  he  or  she  put  into  making  the  choice  on  a  scale  ranging  from  0 
(hardly  any  effort)  to  10  (a  great  deal  of  effort).  The  purpose  of  these  six 
practice  trials  was  threefold:  1)  to  introduce  the  response  scale;  2)  to 
consolidate  the  learning  of  the  rules;  and  3)  to  introduce  subjects  to  the 
range  of  difficulty  in  the  problems  so  that  they  could  calibrate  their  use  of 
the  response  scales  more  accurately  during  the  actual  estimation  sessions. 

This  latter  purpose  was  accomplished  by  selecting  a  variety  of  problem  sizes 
and  difficulty  levels  for  the  six  practice  trials. 

Estimation  Sessions.  At  the  beginning  of  each  session,  the  subject  was 
given  a  review  of  that  session's  decision  rule.  The  rule  was  described  again, 
and  several  practice  trials  were  given,  with  feedback  on  the  accuracy  of  the 
acquisition  sequence  and  choice.  Then  subjects  were  given  a  sequence  of 
decision  problems  where  they  had  to  make  two  consecutive  choices  using  the 
rule  with  no  errors  in  acquisition  sequence  or  alternative  chosen.  Following 
successful  completion  of  these  trials  to  criterion,  the  actual  experimental 
trials  for  that  session  began. 

As  noted  above,  the  twenty  choice  problems  for  each  decision  rule  were 
presented  to  the  subject  on  an  IBM  Personal  Computer  via  the  Mouselab 
software.  Subjects  used  a  Mouse  systems  mouse  as  a  pointing  device.  These 
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problems  were  randomly  ordered  (the  random  order  was  the  same  for  all 
subjects).  For  each  problem,  the  subject  followed  the  sequence  of 
acquisitions  implied  by  the  rule.  The  move  monitoring  system  described  above 
was  used  to  monitor  subjects'  adherence  to  the  correct  sequence  for  the  rule. 
Subjects  then  indicated  the  alternative  chosen,  and  responded  to  the 
difficulty  and  effort  scales  described  above.  For  each  choice,  Mouselab 
recorded  the  sequence  of  acquisitions,  the  time  of  entry  and  exit  for  each 
cell,  the  alternative  chosen,  and  values  on  the  two  response  scales.  The 
overall  latencies  for  the  choice  and  the  two  scale  responses  were  also 
recorded.  Finally,  any  errors  in  acquisition  sequence  were  recorded.  These 
data  were  then  written  to  a  disk  file  for  later  analysis.  This  process  was 
repeated  until  all  twenty  choices  had  been  completed  for  the  given  rule. 

After  completing  all  eight  sessions,  subjects  received  $40  for  their 
participation.  In  addition,  they  were  told  that  three  $5  bonuses  would  be 
paid  for  (l)  above  average  performance  in  terms  of  overall  accuracy,  (2) 
minimization  of  incorrect  search,  and  (3)  speed  of  decision,  respectively.  In 
other  words,  subjects  were  informed  that  they  could  earn  an  additional  payment 
of  up  to  $15  dollars  depending  upon  their  performance. 


Preliminary  Analyses 


Before  the  major  analyses  could  be  performed,  the  data  were  analyzed  to 


determine  the  prevalence  of  errors,  the  existence  of  speed-accuracy  tradeoffs, 


and  the  relationship  between  the  two  self-report  measures  of  effort. 


Subjects  selected  incorrect  alternatives  on  11. 4%  of  the  trials.  In 
addition,  slightly  less  than  1%  (.8%)  of  the  trials  contained  severe 
deviations  from  the  correct  sequence  of  acquisitions  specified  for  that  trial 
(i.e.,  more  than  two  "buzzes"),  even  though  the  correct  alternative  was  still 
selected.  Taken  together,  this  yields  a  total  of  12.2%  error  trials.  Over 
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half  of  these  errors  come  from  the  weighted  adding  (27. 1%)  and  elimination  by 
aspects  (32.2%)  rules.  For  all  analyses,  all  error  trials  of  both  types  were 
removed  from  the  data.  However,  analyses  performed  when  all  trials  were 
included  show  virtually  identical  results. 

To  examine  the  possible  existence  of  speed-accuracy  tradeoffs,  response 
latency  was  correlated  with  error,  both  across  and  within  strategies. 

Overall,  the  correlation  between  time  for  each  decision  and  the  probability  of 
an  error  was  .15  (p<.0001).  Similar  positive  correlations  were  obtained  for 
each  rule,  subject,  and  rule  by  subject  combination.  In  no  case  was  there  a 
significant  negative  correlation,  which  indicated  that  these  data  are 
relatively  free  from  any  concerns  with  speed-accuracy  tradeoffs. 

Finally,  the  two  self-report  measures  of  effort  and  difficulty  were 
examined.  Their  intercorrelation  was  .85,  suggesting  that  they  measure  the 
same  underlying  construct.  A  principal  components  analysis  showed  that  the 
first  factor  accounted  for  93%  of  the  variance  in  the  scores,  so  the  two 
ratings  were  added  to  form  an  overall  index  of  subjective  effort. 

For  the  analyses  we  report,  several  models  are  estimated  using  different 
independent  variables.  In  every  model,  however,  dummy  variables  representing 
the  subject  and  session  (i.e.,  the  order  of  that  session  among  the  six 
estimation  sessions)  are  included,  as  are  variables  representing  the  linear 
and  quadratic  effects  of  trial  (i.e.,  the  order  among  the  twenty  decision 
problems  within  any  session).  These  variables,  although  statistically 
significant,  account  for  small  portions  of  the  explained  variance  and  simply 
allow  for  changes  in  the  intercept  term  across  sessions  and  subjects  and  for 
any  effects  of  practice  across  trials  to  be  taken  into  account.  Since  the 
effects  are  not  theoretically  important  for  our  purposes,  they  are  not 
reported  in  the  discussion  of  the  results. 
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)verview  of  the  Analyses 

As  discussed  above,  we  examined  three  major  indicators  of  strategy 
execution  effort:  response  times,  subjective  reports  of  effort  and 
difficulty,  and  errors.  By  use  of  these  multiple  measures,  we  hope  to  gain 
covergent  evidence  for  the  proposed  approach  to  characterizing  strategy 
execution  effort  in  terms  of  EIP's. 

Recall  that  we  have  described  three  different  classes  of  models  for 
effort:  structural,  behavioral,  and  EIP.  The  structural  model  attempts  to 

describe  effort  solely  in  terms  of  problem  characteristics:  the  number  of 
alternatives  and  attributes,  their  product,  and  a  dummy  variable  for  each 
rule.  The  behavioral  model  attempts  to  explain  effort  using  the  only  overtly 
observable  behavior,  the  number  of  information  acquisitions  (READS).  Finally, 
we  examine  two  different  EIP  models:  the  weighted  EIP  model  attempts  to 
predict  each  dependent  measure  using  as  variables  a  count  for  each  of  the 
EIP's.  Each  mental  oneation  is  thus  allowed  to  have  its  own  characteristic 
effect  upon  the  dependent  measure.  For  example,  each  EIP  has  its  own  latency 
in  the  response  time  analyses.  In  contrast,  the  equal-weighted  EIP  model 
represents  the  null  hypothesis,  in  which  all  EIP’s  are  given  the  same  weight. 
Such  models  have  proven  surprisingly  robust  (Dawes,  1979)  and  provide  a 
reasonable  baseline  for  the  elementary  operations  model.  We  can  assess  the 
relative  fit  of  each  of  these  models  and  test  certain  comparisons. ^  All 
models  also  contain  the  blocking  variables  (Session,  subject  and  trial) 
described  above. 

For  response  time,  we  can  estimate  some  models  at  three  different  levels 


of  aggregation.  The  overall  latency  for  the  decision;  the  latency  for  each 
acquisition;  and,  at  intermediate  levels,  the  total  times  spent  examining  each 
alternative  and  each  attribute.  For  the  overall  decision  times,  all  the 
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models  can  be  estimated,  using  regression  analysis.  At  the  level  of  each 
acquisition,  the  structural  and  behavioral  models  do  not  make  sense;  hence, 
only  the  equal-weight  EIP  and  weighted  EIP  models  are  estimated.^  To  predict 
the  times  spent  on  each  alternative  or  attribute,  the  behavioral  model,  equal- 
weight  EIP  model,  and  weighted  EIP  model  were  estimated. 5  We  examine  multiple 
models  at  several  levels  of  aggregation  because  it  helps  us  deal  with 
intercorrelation  problems  and  also  provides  convergent  evidence  for  the  fit  of 
the  EIP  models  relative  to  the  others  regardless  of  the  level  of  detail  of  the 
analysis  (Card,  Moran,  &  Newell,  1983). 

For  the  self-reports  of  effort,  regression  analyses  were  performed  using 
the  structural,  behavioral,  equal-weight  EIP,  and  weighted  EIP  models,  with 
the  index  of  subjective  effort  as  the  dependent  measure.  Since  this  index  was 
only  meaningful  at  the  level  of  an  individual  decision  problem,  no  analyses  at 
more  disaggregate  levels  were  done. 

Finally,  for  the  error  analyses,  logistic  regressions  (Neter  and 
tfasserman,  1974,  p.  322),  were  used  to  predict  the  probability  that  a  given 
trial  would  produce  an  error.  Since  the  dependent  variable  is  dichotomous  (an 
error  occurred  or  not),  logistic  regression  is  an  appropriate  technique  which 
allows  the  use  of  all  trials  without  having  to  reduce  the  data  to  proportions. 
Since  errors  are  also  only  defined  at  the  level  of  each  decision  problem,  no 
analyses  at  disaggregate  levels  were  performed.  The  structural,  behavioral, 
equal-weight  EIP,  and  weighted  EIP  models  were  run  using  the  occurrence  of  an 
error  or  not  as  the  dependent  measure. 

Table  5  summarizes  these  analyses  across  the  various  dependent  measures, 
models,  and  levels  of  aggregation.  In  all  cases,  the  basic  hypotheses  are 
essentially  the  same.  The  greater  the  number  of  alternatives  and  attributes 
(structural),  reads  (behavioral),  sum  of  EIP's  (equal-weight  EIP),  or  weighted 
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sum  of  EIP's  (weighted  EIP),  then  the  more  time  taken,  greater  subjective 
feelings  of  effort  reported,  and  greater  likelihood  of  execution  error.  Our 
central  hypothesis  is  that  the  EIP  models  will  provide  significantly  better 
fit s  than  the  structural  and  behavioral  models. 

Results 

The  degree  of  fit  for  the  analyses  indicated  in  Table  5  is  summarized  in 
Table  6.  To  provide  the  most  sensitive  tests  of  the  models,  these  analyses 
were  run  by  pooling  across  all  seven  subjects  and  including  dummy  variables 
for  the  subjects.  Since  these  analyses  use  all  of  the  data,  they  provide  the 
most  powerful  tests  for  separating  the  various  models.  However,  this  also 
makes  the  assumption  that  all  subjects  have  identical  latencies  for  each  EIP. 
He  relax  this  assumption  below.  We  examine  the  results  for  each  of  the  three 
major  classes  of  dependent  variables  in  turns  response  time,  subjective 
effort,  and  errors. 

Insert  Table  6  about  here 


Response  Time  Analyses 

Overall  Decision  Times.  Table  6  indicates  that  all  of  the  models 
provide  good  fits  for  the  overall  response  times  (p  <  .0001).  As 
hypothesized,  the  weighted  EIP  model  provides  the  best  degree  of  fit,  R2  * 

.81.  The  fit  of  the  weighted  EIP  model  is  significantly  better  than  that  of 
the  behavioral  model  (F(5,  713)  »  77.8,  p  <  .0001)  or  that  of  the  equal~weight 
EIP  model  (F(5,  713)  *  74.6,  p  <  .0001). ^  While  the  structural  and  weighted 
EIP  models  cannot  be  directly  compared  statistically,  it  is  clear  that  the 
weighted  EIP  model  provides  a  better  degree  of  fit.  In  addition,  if  we  add 
the  variables  from  the  structural  model  (problem  size  and  rule  dummy 
variables)  to  the  weighted  EIP  model,  the  expanded  model  does  not  provide 
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improved  fit,  with  an  incremental  R^  ■  .005,  n.s.  Finally,  these  results  hold 
up  well  in  cross  validation.  Estimating  the  model  on  one-half  of  the  data  and 
using  these  estimates  to  predict  the  other  half  yields  average  R^  of  .63,  .70, 
.68,  and  .89  for  the  structural,  behavioral,  equal-weight  EIP,  and  weighted 
EIP  models,  respectively. ^ 

Individual  Acquisition  Times.  Given  the  success  of  the  weighted  EIP 
model  at  the  level  of  overall  decision  times,  we  turn  to  other  levels  of 
analysis  for  evidence  that  the  results  obtained  are  consistent  across  levels 
of  aggregation.  However,  as  Card,  Moran,  and  Newell  (1983)  argue,  analyses  at 
more  disaggregate  levels  will  tend  to  have  lower  absolute  levels  of  fit,  for 
several  reasons:  measurement  of  the  exact  operators  employed  is  likely  to  be 
more  errorful  at  lower  levels  of  aggregation,  and  variation  in  the  time 
required  to  execute  operators  will  be  more  apparent  when  the  number  of  EIP's 
for  each  case  is  smaller,  as  in  these  more  disaggregate  analyses.  For 
example,  while  we  assume  that  all  additions  take  the  same  amount  of  time,  it 
is  clear  that  some  additions  (i.e.  2+2)  take  less  time  than  others  (i.e.  8+7). 
This  assumption  is  most  problematic  at  the  more  disaggregate  level,  where  the 
characteristics  of  a  single  addition  largely  determine  the  latency. 

In  examining  the  fit  of  the  equal-weight  and  weighted  EIP  models  at  the 
individual  acquisition  level,  also  shown  in  Table  6,  it  is  apparent  that  the 
absolute  level  of  fit  has  indeed  declined,  although  it  is  still  significant  at 
p  <  .0001.  The  weighted  cognitive  model  (R^  ■  .23)  still  performs  better  than 
the  equal-weight  cognitive  model  (R^  ■  .16)  (F(4,  12159)  ■  325.30,  p  <  .0001). 

Times  for  Each  Alternative  and  Attribute.  The  models  estimated  to 
predict  the  total  time  spent  on  each  alternative  and  the  total  time  spent  on 
each  attribute  replicate  those  reported  above  (See  Table  6).  The  behavioral 
and  equal-weight  EIP  models  perform  roughly  equally,  and  both  provide 
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significantly  lower  fits  than  the  weighted  EIP  model.  For  time  spent  on 
alternatives,  the  weighted  EIP  model  is  superior  to  the  behavioral  and  equal- 
weight  EIP  models,  F<5,  2943)  -  297.54,  p  <  .0001  and  F(5,  2943)  -  351.02,  p  < 
.0001,  respectively.  For  time  spent  on  attributes,  the  same  is  true,  F(5, 
2313)  -  204.39,  p  <  .0001  and  F(5,  2313)  -  178.86,  p  <  .0001,  respectively. 

More  Complex  Models.  These  analyses  strongly  and  consistently  support 
the  proposed  EIP  model  for  measuring  strategy  execution  effort.  The  weighted 
EIP  model  provides  the  greatest  degree  of  fit  across  four  levels  of 
aggregation  in  response  times.  The  degree  of  fit  at  the  overall  decision  time 
level  is  quite  impressive,  given  the  complexity  of  the  decision  tasks  studied. 
In  addition,  the  model  comparisons  allow  several  conclusions:  1)  Models  which 
examine  structural  characteristics  of  the  task  (e.g.,  problem  size  and  rule 
dummy  variables)  are  not  sufficient;  2)  Models  using  observable  behaviors 
(acquisitions)  alone  do  not  suffice,  so  a  more  complete  set  of  EIP's  is 
necessary;  3)  Each  EIP  should  be  allowed  to  have  its  own  weight,  as  the 
weighted  EIP  model  consistently  outperforms  the  equal-weighted  model. 

Since  the  weighted  EIP  model  seems  necessary,  in  the  sense  that  it 
provides  superior  fit  to  its  three  simpler  competitors,  we  also  examine 
whether  it  is  sufficient.  That  is,  will  more  complex  models  improve  on  the 
degree  of  fit?  These  models  are  reported  only  for  the  overall  decision  time 
analysis,  but  were  replicated  at  other  levels  with  similar  results.  The  first 
complex  model  directly  tested  the  hypothesis  that  the  EIP's  require  the  same 
time  for  each  rule.  This  is  tested  by  allowing  the  time  taken  by  each  EIP  to 
vary  from  rule  to  rule.  The  weighted  EIP  model  is  then  a  special  case  of  this 
augmented  model,  and  the  significance  of  the  incremental  fit  can  be  tested. 
Although  the  incremental  fit  is  significant  (F(13,  700)  »  1.91,  p  <  .05),  the 
incremental  is  very  small,  .007.  Hence,  as  a  first  approximation,  the 


Strategy  Effort 
30 

assumption  that  each  operation  requires  a  constant  amount  of  time  independent 
of  the  strategy  in  which  it  is  used  seems  reasonable. 

The  second  more  complex  model  allows  the  times  for  the  EIP's  to  vary 
across  subjects.  Even  if  individuals  use  the  same  strategy,  they  may  differ 
in  the  amount  of  time  required  for  each  component  process  (Hunt,  1978;  R. 
Sternberg,  1977).  This  model  achieves  an  ■  .90,  with  significantly  better 
fit  than  the  weighted  cognitive  model  (Incremental  R^  *  .09,  F(19,  674)  ■ 

17.9,  p  <  .001). 8  Given  the  small  number  of  individuals  in  the  sample,  these 
individual  differences  are  not  pursued  further. 

Thus,  based  upon  the  analyses  of  response  times,  the  weighted  EIP  model, 
and  hence  the  EIP  conceptualization  of  decision  effort,  receives  strong 
support.  The  EIP  times  appear  to  vary  across  individuals,  although  not  across 
rules.  Moreover,  the  weighted  EIP  model  provides  the  best  fit  across  all  of 
the  various  analyses  attempted. 

Estimates  of  EIP  Times.  Since  the  weighted  EIP  model  received  strong 
support,  estimates  of  the  times  for  each  operator  are  shown  in  Table  7. 
Although  the  estimates  vary  to  some  extent  across  individuals,  as  noted  above, 
we  will  examine  the  pooled  results  for  the  different  levels  of  aggregation. 

Insert  Table  7  about  here 

The  estimates  are  generally  consistent  across  levels,  which  provides 
increased  confidence  in  the  values.  The  coefficients  are  all  positive,  with 
most  significantly  so.  The  estimates  also  tend  to  agree  with  estimates  for 
similar  EIP's  provided  by  other  studies.  Focusing  upon  the  overall  decision 
time  analysis,  we  now  consider  the  estimates  of  each  operator. 

The  READ  EIP  combines  encoding  information  with  the  motor  activity  of 
moving  the  mouse.  Its  estimated  latency  is  1.19  seconds  (t(713)  *  6.55,  p  < 
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.0001).  This  estimate  is  plausible,  since  it  might  consist  of  the  movement  of 
the  mouse,  estimated  to  be  in  the  range  of  .2  -  .8  seconds  by  Johnson,  Payne, 
Schkade,  and  Bettman  (1986),  and  an  eye  fixation,  estimated  to  require  a 
minimum  of  .2  seconds  (Russo,  1978).  ADDITIONS  and  SUBTRACTIONS  both  take 
less  than  one  second,  with  estimates  of  .84  (t(713)  ■  4. 54,  p  <  .0001)  and  .32 
(t(713)  •  .98,  n.s.)  respectively.  These  values  are  not  significantly 
different  (t(713)  ■  1.03,  n.s.)  and  are  consistent  with  those  provided  by 
Dansereau  (1969),  Groen  and  Parkman  (1972),  and  others  (see  Chase,  1978,  Table 
3,  p  .76).  Our  estimate  for  the  PRODUCT  EIP,  2.23  seconds  (t(713)  «  10.36,  p 
<  .0001),  is  larger  than  that  commonly  reported  in  the  literature. 

The  time  for  COMPARISONS  is  very  short,  .08  seconds  (t(713)  *  .22, 
n.s.),  and  that  for  ELIMINATIONS,  1.80  seconds  (t(713)  -  3.00,  p  <  .01),  is 
relatively  long.  This  may  reflect  the  collinearity  of  COMPARES  and 
ELIMINATIONS  at  the  overall  decision  time  level.  There  is  some  suggestive 
evidence  for  this  in  the  results  at  the  more  disaggregate  levels.  Although 
the  estimates  for  READS,  ADDITIONS,  PRODUCTS,  and  DIFFERENCES  are  fairly 
similar  across  levels,  the  COMPARISON  and  ELIMINATION  estimates  vary  widely. 
Since  the  intercorrelation  between  COMPARISONS  and  ELIMINATIONS  is  no  higher 
than  .51  for  these  more  disaggregate  analyses,  it  is  likely  that  the  lower 
estimates  for  ELIMINATIONS  and  higher  estimates  for  COMPARISONS  found  in  the 
disaggregate  analyses  are  more  plausible. 

In  sum,  based  both  upon  its  degree  of  fit  and  the  generally  plausible 
time  estimates  for  the  EIP's,  the  proposed  weighted  EIP  model  receives 
impressive  support  when  response  times  are  used  as  an  indicator  of  effort. 

The  next  set  of  results  examines  the  performance  of  the  various  models  when 
subjective  effort  reports  form  the  indicator  of  effort. 
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Subjective  Effort  Analyses 

There  are  several  reasons  why  subjective  reports  of  effort  are 
interesting  as  a  second  indicator  of  decision  effort.  First,  subjective 
effort  might  tap  different  aspects  of  strategy  execution  effort  and  might  not 
be  closely  related  to  decision  latency.  As  Kahneman  (1973)  observed,  two 
different  mental  tasks  may  take  similar  amounts  of  time,  but  one  might  be  seen 
as  much  more  effortful  than  the  other.  This  speculation  receives  support  in 
our  data:  the  overall  correlation  between  time  and  the  subjective  effort 
index  is  .29.  Secondly,  while  the  analysis  of  latency  helps  validate  the 
proposed  EIP  conceptualization  of  effort,  subjective  perceptions  of  effort  may 
be  important  in  understanding  why  decision-makers  avoid  certain  strategies. 

If  certain  mental  operations  are  perceived  to  be  more  effortful  than  others, 
such  operations  may  well  be  avoided  in  decision-making.  Associating  a  typical 
perception  of  effort  with  these  EIP's  may  help  us  better  understand  strategy 
choice.  Several  cogent  arguments  for  caution  in  the  use  of  subjective 
measures  of  effort  should  also  be  noted.  Foremost  among  these  is  the 
possibility  that  subjects  cannot  accurately  report  demands  on  cognitive 
resources  (Gopher  and  Donchin,  1986),  or  that  such  reports  do  not  allow 
comparisons  across  tasks  which  make  widely  differing  demands. 

Model  Fit.  From  the  results  shown  in  Table  6,  it  can  be  seen  that  the 
weighted  EIP  model  again  provides  the  best  fit  to  the  subjective  effort  data. 
The  absolute  levels  of  fit  are  lower  than  for  the  overall  response  latencies, 
but  are  still  highly  significant  (p  <  .0001).  The  weighted  EIP  model  provides 
significantly  greater  fit  than  the  behavioral  (F(5,  717)  »  10.52,  p  <  .001) 
and  equal-weight  EIP  (F(5,  717)  ■  13.32,  p  <  .001)  models. 

The  weighted  EIP  model  of  subjective  effort  can  also  be  compared  to  more 


complex  models.  Adding  the  structural  variables  (problem  size  and  rule 
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dummies)  does  not  improve  the  fit  (incremental  *  .007,  n.s.).  However,  as 
was  the  case  for  the  response  time  analysis,  allowing  the  effort  for  each 
operation  to  vary  for  each  rule  produces  a  small,  but  statistically 
significant,  increase  in  fit  (incremental  R^  ■  .01,  F(13,  704)  ■  2.47,  p  < 
.01).  Finally,  and  again  very  similar  to  the  case  for  response  times, 
allowing  the  effort  estimates  for  each  EIP  to  vary  across  individuals 
significantly  improves  the  degree  of  fit  (R^  *  .80,  p  <  .0001). 

Hence,  the  results  essentially  replicate  those  for  response  times.  The 
weighted  EIP  model  provides  the  best  explanation  of  decision-makers'  self 
reports  of  the  effort  associated  with  each  decision  problem,  and  the  effort 
estimates  appear  to  vary  across  individuals,  but  only  slightly  across  rules. 

Estimates  of  EIP  Effort.  Estimates  of  the  subjective  effort  associated 
with  each  EIP  from  the  weighted  cognitive  model  pooled  across  subjects  are 
given  in  Table  8(.  These  estimates  represent  the  increase  in  reported  effort 
per  EIP  on  the  sum  of  two  0-10  scales.  The  largest  estimate  is  for  the 
ELIMINATION  operator,  .32.  However,  the  high  intercorrelation  between 
ELIMINATIONS  and  COMPARISONS  (.85)  must  temper  any  interpretation  of  this 
coefficient  and  the  small  (.04)  coefficient  for  COMPARISONS.  The  PRODUCT 
operator,  as  might  be  expected,  is  seen  as  fairly  effortful,  with  a 
coefficient  of  .19,  while  the  coefficients  for  READS  and  ADDITIONS  are  also 
significantly  positive.  These  coefficients  suggest  that  the  perceived  effort 
associated  with  various  EIP's  may  vary  widely,  which  would  help  to  explain  why 
some  strategies  are  perceived  more  favorably  than  others  in  situations  where 
subjects  are  free  to  select  their  own  strategies. 

Error  Analyses 


A  final  indicator  of  strategy  execution  effort  is  errors  in  strategy 
execution,  under  the  hypothesis  that  greater  required  effort  leads  to  a  higher 
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probability  of  making  an  execution  error.  The  logistic  regression  results  for 
the  error  analysis  are  given  in  Table  6.  The  pseudo-R^  reported  in  the  table 
is  analogous  to  an  F-test  in  regression,  and  nested  tests  of  competing  models 
can  also  be  conducted  using  tests  for  differences  in 

From  the  results,  we  see  that  the  weighted  EIP  model  again  performs 
better  than  the  behavioral  model  (X^(5)  ■  11.43,  p  <  .05)  and  marginally 
better  than  the  equal-weight  EIP  model  (X^(5)  *  9.15,  p  ■  .105).  This  largely 
replicates  the  findings  for  time  and  subjective  effort.  However,  and  in 
contrast  to  those  earlier  findings,  the  structural  model  performs  better  than 
the  weighted  EIP  model  in  terms  of  fit.  In  addition,  adding  the  structural 
variables  to  the  weighted  EIP  model  improves  its  fit  (R^  ■  .69,  X^(8)  “  27.87, 
p  <  .001).  Thus,  errors  appear  to  depend  not  only  on  the  proposed  measure  of 
strategy  execution  effort  but  on  the  size  of  the  choice  problem.  Hence, 
errors  may  involve  aspects  of  strategy  execution  not  directly  captured  by  the 
ElP's  considered  here.  In  particular,  these  EIP's  may  not  capture  various 
"bookkeeping"  aspects  of  decision  strategies,  such  as  remembering  the  current 
best  or  which  alternatives  and  attributes  have  been  eliminated.  Such  aspects 
may  increase  with  the  size  of  the  problem  and  lead  to  greater  likelihood  of  an 
execution  error. 

Discussion 

The  concept  of  effort  plays  a  major  role  in  attempts  to  understand  the 
contingent  use  of  processing  strategies.  An  approach  to  measuring  the  effort 
associated  with  different  decision  strategies  is  proposed  in  this  study,  using 
a  set  of  elementary  operators  (i.e.,  READS,  ADDITIONS,  COMPARISONS,  PRODUCTS, 
DIFFERENCES,  and  ELIMINATIONS)  as  a  common  "language"  for  describing  decision 
strategies.  This  is  used  to  generate  a  metric  of  the  effort  required  to 
execute  a  decision  strategy  in  terms  of  the  number  of  EIP's  involved. 
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The  empirical  results  yielded  strong  support  for  this  proposed 
componential  approach  to  strategy  effort.  A  model  of  effort  based  upon 
weighted  EIP  counts  (the  weighted  EIP  model)  was  found  to  provide  the  best 
predictions  of  response  times  at  several  different  levels  of  aggregation  and 
of  self-reports  of  effort,  two  different  measures  of  decision  effort.  The 
weighted  EIP  model  also  provided  good  fits  to  another  indicator  of  effort, 
error  data.  In  addition  to  this  absolute  level  of  fit,  the  weighted  EIP  model 
also  was  statistically  superior  to  a  behavioral  model  using  only  reads  and  to 
an  equal-weight  EIP  model  for  each  of  the  three  Indicators  of  effort.  The 
weighted  EIP  model  was  also  superior  to  a  structural  model  using  predictors 
based  upon  problem  size  for  the  response  time  and  self-reported  effort  data, 
although  the  structural  model  was  superior  for  modeling  errors.  Taken 
together,  these  results  imply  that  a  small  number  of  simple  operators  can  be 
viewed  as  the  fundamental  components  from  which  decision  rules  are  constructed 
(Bettman  and  Park,  1980).  Whether  the  current  set  of  proposed  operators  is 
sufficient  is  open  to  debate,  based  upon  the  error  results,  but  the  important 
point  is  that  an  EIP  approach  seems  highly  promising. 

In  addition  to  the  support  obtained  from  the  overall  levels  of  fit,  the 
estimates  of  time  taken  for  each  EIP  were  mostly  plausible  and  in  line  with 
prior  research,  hence  providing  additional  confidence  in  the  approach. 

Similar  estimates  also  generally  emerge  as  the  analysis  is  repeated  at 
different  levels  of  aggregation.  Significantly,  the  estimates  also  appear  to 
remain  essentially  the  same  regardless  of  the  strategy  used.  However,  there 
do  appear  to  be  significant  individual  differences  in  the  times  taken  for  the 
individual  EIP's.  For  example,  for  some  individuals,  arithmetic  operators  may 
be  relatively  more  difficult  than  comparisons;  for  others  the  difference  may 
be  less  pronounced  or  even  reversed.  This  implies  that  individuals  may  choose 
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ifferent  rules  in  part  because  the  different  component  EIP's  may  be 
-elatively  more  or  less  difficult  or  effortful  across  individuals.  Although 
:he  number  of  subjects  was  too  small  to  consider  these  issues  in  the  current 
research,  they  offer  intriguing  possibilities  for  future  research. 

Another  contribution  of  the  study  is  more  methodological.  The  Mouselab 
decision-monitoring  software  and  hardware  worked  exceptionally  well  in 
providing  detailed  data  about  the  decision  task.  The  ability  to  monitor  the 
sequence  of  acquisitions,  measure  latencies,  and  in  general  maintain 
experimental  control  over  the  choice  task  makes  this  system  potentially  very 
valuable  for  a  variety  of  research  issues  in  decision  making  and  other  areas 
of  cognition. 

The  attainment  of  experimental  control,  necessary  to  predict  the 
operators  used  and  implement  the  proposed  EIP  models  of  effort,  is  not  without 
costs.  Subjects  do  not  select  strategies;  rather,  they  apply  given  rules. 
Hence,  the  task  eliminates  many  difficult  problems  normally  faced  by 
individuals  making  decisions.  Subjects  did  not  have  to  select  or  construct  a 
strategy,  and  the  sequence  of  operations  was  specified.  Thus,  they  did  not 
have  to  engage  in  possibly  effortful  control  processes  determining  what  to  do 
next.  In  addition,  by  providing  all  of  the  weights,  cutoffs,  and  ratings,  the 
need  for  potentially  difficult  valuation  processes  was  eliminated.  Finally, 
some  of  the  timing  estimates  are  undoubtedly  affected  by  the  specific 
apparatus  used  (i.e.,  the  matrix  display  and  the  mouse).  Further  research 
relaxing  these  restrictions  on  processing  flexibility  would  be  desirable. 
However,  maintaining  sufficient  experimental  control  is  essential  for  research 
at  this  level  of  detail  regarding  decision  effort. 

A  second  set  of  caveats  is  that  although  an  approach  which  breaks  down 
decision  strategies  into  more  detailed  components  seems  to  be  strongly 
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supported  as  an  approach  to  measuring  decision  effort,  we  have  focused  on  a 
particular  level  of  detail  in  taking  such  an  approach.  For  example,  one  could 
sodel  multiplications  in  terms  of  underlying  arithmetic  operations  (e.g., 

Danse reau,  1969;  Lopes,  1982).  In  addition,  one  could  extend  our  models  to 
Include  EIP's  that  model  short-term  memory  load  and  other  mental  "bookkeeping" 
operations.  The  error  analyses  provide  a  hint  that  such  an  extension  to  the 
aodels  used  would  be  fruitful,  but  a  demonstration  awaits  future  research. 

The  proposed  weighted  EIP  model  appears  to  provide  a  good  approach  to 
neasuring  decision  effort.  To  examine  strategy  selection,  however,  one  must 
also  consider  the  accuracy  of  a  strategy  in  terms  of  the  goodness  of  the 
choices  made.  Since  strategies  were  specified  in  the  current  study,  not 
;hosen,  there  were  no  data  available  on  accuracy  in  the  above  sense.  However, 
:he  effort  models  can  provide  important  input  regarding  accuracy-effort 
rradeoffs.  The  EIP  time  estimates,  for  example,  could  be  used  to  model  effort 
in  simulation  studies  which  have  examined  accuracy-effort  tradeoffs  (e.g., 
Johnson  and  Payne,  1985;  Payne,  Bettman,  and  Johnson,  1986).  These  estimates 
night  also  be  used  to  predict  strategy  selection  across  decision  problems  or 
individuals,  particularly  if  individuals  were  trained  on  a  variety  of 
strategies  and  constrained  to  select  one.  These  selections  could  be  modeled 
>ased  upon  predicted  effort,  the  relative  accuracy  of  the  strategies,  and  any 
experimental  variables  affecting  the  weights  given  to  accuracy  versus  effort. 

The  approach  to  measuring  cognitive  effort  developed  in  this  paper  may 
llso  have  applied  value.  For  example,  recently  it  has  been  suggested  that  the 
ise  of  nutritional  information  in  the  supermarket  by  consumers  might  be 
.mproved  by  decreasing  the  effort  costs  associated  with  processing  that 
.nformation  (Russo,  Staelin,  Nolan,  Russell,  and  Metcalf,  1986).  The 
lethodology  developed  in  this  paper  could  be  used  to  test  the  impact  of 
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different  information  displays  on  the  use  of  a  preferred  decision  strategy.  A 
related  area  of  application  would  be  the  design  of  computer-based  decision 
aids  (Keen  &  Scott-Morton,  1978). 

As  a  final  point,  research  on  decision  making  is  a  field  of  study  that 
has  drawn  from  cognitive  psychology,  but  has  not  been  as  integrated  into 
cognitive  psychology  as  it  might  be  (See  Pitz,  1977,  for  a  discussion  of  why 
such  a  schism  may  exist).  The  issues  of  measuring  strategy  execution  effort 
and  contingent  strategy  use  in  making  a  decision  or  solving  a  problem, 
however,  are  ones  where  the  two  fields  might  benefit  from  a  closer 
interaction.  The  present  investigation  draws  upon  ideas  of  decomposition, 
chronometric  techniques,  and  error  analysis  that  have  proved  successful  in 
understanding  performance  in  a  variety  of  cognitive  tasks.  On  the  other  hand, 
characterizing  factors  affecting  strategy,  effort,  accuracy,  and  selection 
across  individuals  and  problem  characteristics  in  various  cognitive  tasks  may 
benefit  from  the  extensive  work  on  contingent  decision  strategies  that 
currently  exists.  The  results  of  the  current  research  on  modeling  decision 
effort  demonstrate  that  such  a  partnership  can  be  highly  fruitful.  Hopefully, 
this  will  encourage  further  research  efforts  of  this  sort. 


References 


Strategy  Effort 
39 


ach,  L.  R. ,  &  Mitchell,  T.  R.  (1978).  A  contingency  model  for  the  selection 
of  decision  strategies.  Academy  of  Management  Review,  3,  439-449. 

:tman,  J.  R. ,  &  Park,  C.  W.  (1980).  Effects  of  prior  knowledge,  experience, 
and  phase  of  the  choice  process  on  consumer  decision  processes:  A 
protocol  analysis.  Journal  of  Consumer  Research,  234-248. 
ttman,  J.  R. ,  &  Zins,  M.  (1979).  Information  format  and  choice  task  effects 
in  decision  making.  Journal  of  Consumer  Research,  6,  141-153. 

3wn,  A.  L.,  Bransford,  J.  D.,  Ferrara,  R.  A.,  &  Campion! ,  J.  C.  (1983). 
Learning,  remembering,  and  understanding.  In  P.  H.  Mussen  (Ed.),  Handbook 
of  Child  Psychology:  Cognitive  Development,  Volume  3.  New  York:  John 
Wiley. 

rd,  S.  K.,  Moran,  T.  P. ,  &  Newell,  A.  (1983).  The  psychology  of  human- 
computer  interaction.  Hillsdale,  NJ:  Lawrence  Erlbaum. 
rpenter,  P.  A.,  &  Just,  M.  A.  (1975).  Sentence  comprehension:  A 
psycholinguistic  processing  model  of  verification.  Psychological  Review, 
82,  45-73. 

ase,  W.  G.  (1978).  Elementary  information  processes.  In  W.  K.  Estes  (Ed.), 
Handbook  of  Learning  and  Cognitive  Processes,  Volume  5.  Hillsdale,  NJ: 
Lawrence  Erlbaum. 

nsereau,  D.  F.  (1969).  An  information  processing  model  of  mental 
multiplication.  Unpublished  dissertation,  Camegie-Mellon  University. 

»es,  R.  M.  (1979).  The  robust  beauty  of  improper  linear  models  in  decision 
making.  American  Psychologist,  34,  571-582. 

pher,  D.,  &  Donchin,  E.  (In  press).  Workload:  An  examination  of  the 
concept.  In  K.  Boff  &  L.  Kaufman  (Eds.),  Handbook  of  Perception  and  Human 
Performance .  New  York:  John  Wiley. 


Strategy  Effort 
40 

3roen,  G.  J.,  &  Parkman,  J.  M.  (1972).  A  chronometric  analysis  of  simple 
addition.  Psychological  Review,  79,  329-343. 

Buber,  0.  (1980).  The  Influence  of  some  task  variables  on  cognitive 

operations  In  an  information-processing  decision  model.  Acta 
Psychologies.  45,  187-196. 

Hunt,  E.  B.  (1978).  Mechanics  of  verbal  ability.  Psychological  Review,  85. 
109-130. 

Johnson,  E.  J.  (1979).  Deciding  how  to  decide:  The  effort  of  making  a 
decision.  Unpublished  manuscript.  University  of  Chicago. 

Johnson,  E.  J.,  &  Payne,  J.W.  (1985).  Effort  and  accuracy  in  choice. 
Management  Science.  31,  394-414. 

Johnson,  E.  J.,  Payne,  J.  W. ,  Schkade,  D.  A.,  &  Bettman,  J.  R.  (1986). 
Monitoring  information  processing  and  decisions:  The  mouselab  system. 
Unpublished  manuscript,  Fuqua  School  of  Business,  Duke  University. 

Kahneman,  D.  (1973).  Attention  and  effort.  Englewood  Cliffs,  NJ:  Prentice- 
Hall. 

Keen,  P.  G.  W. ,  &  Scott-Morton,  M.  S.  (1978).  Decision  support  systems:  An 
organizational  perspective.  Reading:  Mass.:  Addison-Wesley. 

Klayman,  J.  (1985).  Children's  decision  strategies  and  their  adaptation  to 
task  characteristics.  Organizational  Behavior  and  Human  Decision 
Processes,  35,  179-201. 

Kmenta,  J.  (1971).  Elements  of  econometrics.  New  York:  Macmillan. 

Lopes,  L.  L.  (1982).  Toward  a  procedural  theory  of  judgment.  Unpublished 
manuscript,  University  of  Wisconsin. 


Navon,  D.,  &  Gopher,  D.  (1979).  On  the  economy  of  the  human  processing 


Tv  vr  tv  '-rj  w>i  w  w/w? 


T^T 


»■.'»  r*r^r 'n^iwi 


K' 


*• 

> 

* 

>i 

f\ 


£ 

I 


Strategy  Effort 
41 

eter,  J. ,  &  Vasserman,  W.  (1974).  Applied  linear  statistical  models. 
Homewood,  Ill.:  Richard  D.  Irwin. 

ewell.  A.,  &  Simon,  H.  A.  (1972).  Human  problem  solving.  Englewood  Cliffs, 
NJ:  Prentice-Hall. 

'ayne,  J.  W.  (1976).  Task  complexity  and  contingent  processing  in  human 
decision  making:  An  information  search  and  protocol  analysis. 
Organizational  Behavior  and  Human  Performance,  16,  366-387. 

’ayne,  J.  W.  (1982).  Contingent  decision  behavior.  Psychological  Bulletin, 
92,  382-402. 

’ayne,  J.  W. ,  Bettman,  J.  R. ,  &  Johnson,  E.  J.  (1986).  Adaptive  strategy 
selection  in  decision  making.  Unpublished  manuscript,  Fuqua  School  of 
Business,  Duke  University. 

’itz,  G.  F.  (1977).  Decision-making  and  cognition.  In  H.  Jungermann  &  G.  de 
Zeeuw  (Eds.),  Decision-Making  and  Change  in  Human  Affairs.  Dordrecht, 
Holland:  Reidel. 

leder,  L.  M.  (1982).  Plausibility  judgments  versus  fact  retrieval: 

Alternative  strategies  for  sentence  verification.  Psychological  Review, 
89,  250-280. 

Russo,  J.  E.  (1978).  Eye  fixations  can  save  the  world:  Critical  evaluation 
and  comparison  between  eye  fixations  and  other  information  processing 
methodologies.  In  H.  K.  Hunt  (Ed.),  Advances  in  Consumer  Research,  Volume 
V.  Ann  Arbor,  Michigan:  Association  for  Consumer  Research. 

Russo,  J.  E.,  &  Dosher,  B.  A.  (1983).  Strategies  for  multiattribute  binary 
choice.  Journal  of  Experimental  Psychology:  Learning,  Memory,  and 
Cognition,  9,  676-696. 


-<^r. 


*  »*» 


I  **  »*  V  m*  «•  , 


’v’v.v;/**] 


Authors '  Notes 


Strategy  Effort 
43 

The  research  reported  in  this  paper  was  supported  by  a  contract  from  the 
Engineering  Psychology  Program  of  the  Office  of  Naval  Research.  The  order  of 
authorship  is  arbitrary.  Each  author  contributed  equally  to  all  phases  of 
this  project.  Requests  for  reprints  should  be  sent  to  James  R.  Bettman, 

Center  for  Decision  Studies,  Fuqua  School  of  Business,  Duke  University, 

Durham,  North  Carolina  27706. 


Footnotes 


Strategy  Effort 
44 

*Note  that  the  adjustment  of  values  by  probabilities  or  decision  weights 
implied  by  the  PRODUCT  operator  in  Table  1  may  not  involve  a  literal 
multiplication  of  two  quantities;  rather,  they  may  be  combined  by  some  more 
basic  analogical  process  which  adjusts  the  value  of  one  quantity  given  another 
(Lopes,  1982).  For  computer  simulations  of  choice  strategies  as  production 
systems  based  upon  collections  of  such  EIPs,  see  Johnson  and  Payne  (1985). 

^Another  advantage  of  the  decomposition  approach  to  decision  strategies 
is  that  it  expresses  choice  strategies  using  elementary  processes  similar  to 
those  used  for  describing  other  cognitive  tasks.  If  EIP's  possess  essentially 
the  same  properties  across  a  variety  of  problem  solving  tasks,  the  integration 
of  decision  research  with  research  in  other  areas  of  cognitive  psychology 
would  be  facilitated.  Chase  (1978)  provides  a  more  general  discussion  of 
usiag  the  EIP  concept  in  the  analysis  of  information  processing  across  a 
variety  of  cognitive  tasks. 

The  behavioral  model  and  the  equal-weight  EIP  model  are  special  cases 
of  (or  nested  within)  the  weighted  EIP  model.  Hence,  the  additional  fit 
provided  by  the  weighted  EIP  model  over  each  of  these  two  simpler  models  can 
be  tested  statistically  (Ne ter  and  Vasserman,  1974,  p.  89). 

*For  the  individual  acquisitions,  the  EIP’s  involved  can  be  predicted. 
For  example,  in  the  weighted  adding  example  provided  above,  when  a  rating  is 
acquired,  a  READ,  a  PRODUCT,  and  possibly  an  ADDITION  (if  it  is  not  the  first 
attribute)  are  performed.  Since  every  acquisition  has  a  READ,  the  behavioral 
model  cannot  be  used,  and  the  READ  operator  cannot  be  used  in  the  weighted  EIP 
model.  Also,  it  is  not  clear  how  to  relate  the  overall  problem  size  to 
predictions  for  each  acquisition. 
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^Since  these  analyses  aggregate  over  different  numbers  of  acquisitions 
}r  each  problem  size  and  rule,  the  behavioral  model  can  again  be  estimated 

6 The  degrees  of  freedom  for  the  numerator  in  these  comparisons  represent 
ie  difference  between  the  use  of  six  EIP  variables  for  the  weighted  E1P  model 
ad  one  variable  for  the  behavioral  and  equal-weight  EIP  models.  The  degrees 
f  freedom  for  the  denominator  reflect  the  total  trials  and  the  total  number 
f  variables  used  for  the  weighted  EIP  model  (Meter  and  Vasserman,  1974,  p. 

9). 

? All  of  these  analyses  were  repeated  deleting  the  observations  with  the 
argest  latencies.  The  results  were  essentially  identical. 

^Models  of  overall  decision  times  were  run  for  each  of  the  seven 
ubjects,  with  degrees  of  fit  ranging  from  .72  to  .97.  The  average  degree  of 
it  was  .71,  .77,  .77,  and  .89  for  the  structural,  behavioral,  equal-weight 
IP,  and  weighted  EIP  models  respectively.  Thus,  these  results  essentially 
eplicate  the  analyses  pooling  across  subjects  reported  in  Table  6. 


Table  1 
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Elementary  EIP's  Used  in  Decision  Strategies 


READ 

COMPARE 

DIFFERENCE 

ADD 

PRODUCT 

ELIMINATE 

MOVE 


Read  an  alternative's  value  on  an  attribute  Into  STM 
Compare  two  alternatives  on  an  attribute 

Calculate  the  size  of  the  difference  of  two  alternatives  for 
an  attribute 

Add  the  values  of  an  attribute  in  STM 
Height  one  value  by  another  (Multiply) 

Remove  an  alternative  or  attribute  from  consideration 
Go  to  next  element  of  external  environment 


CHOOSE 


Announce  preferred  alternative  and  stop  process 


Table  2 
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Example  of  a  Four  Alternative,  Three  Attribute  Decision  Problem 


Attributes 

Alternatives 

Leadership 

Creativity 

Experience 

tfelghts 

6(1) 

4(2) 

2(3) 

A 

4(4) 

7(5) 

4(6) 

B 

2(7) 

7(8) 

2(9) 

C 

6(10) 

6(11) 

3(12) 

D 

5(13) 

7(14) 

2(15) 

>.v.'  i  -/Of o/n  ■**  iu^ii 


Table  3 
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Example  of  a  Three  Alternative,  Four  Attribute  Decision  Problem 


Attributes 


Alternatives 

Leadership 

Motivation 

Creativity 

Experience 

Weights 

4(1) 

5(2) 

3(3) 

6(4) 

Cutoffs 

7(5) 

4(6) 

6(7) 

6(8) 

A 

6(9) 

5(10) 

7(11) 

7(12) 

B 

7(13) 

4(14) 

3(15) 

6(16) 

C 

4(17) 

3(18) 

4(19) 

4(20) 

Table  4 
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Intercorrelations  Among  EIP  Counts  for  the  120  Decision  Problems  Selected 


ADDITIONS 

PRODUCTS 

COMPARES 

Operators 

ELIMINATIONS 

DIFFERENCES 

READS 

.487 

.543 

.541 

.280 

.272 

ADDITIONS 

.591 

-.259 

-.495 

.140 

PRODUCTS 

-.302 

-.374 

-.146 

COMPARES 

.852 

.492 

ELIMINATIONS 


158 


Table  S 
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Summary  of  Analyses  Performed8 


Models 

Dependent  Variable  Structural  Behavioral  Equal-Weight  EIP  Weighted  E1P 


Response  Times 

Overall  Decision  X  X 

Individual  Acquisition 
Time  on  Each  Alternative  X 

Time  on  Each  Attribute  X 

Index  of  Subject  Effort  X  X 


X 

X 

X 

X 

X 


X 

X 

X 

X 

X 


Errors 


X 


X 


X 


X 


Table  6 
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S«iwwj»ry  of  Model  Fit  Statistics 


Models 

Structural  Behavioral  Equal-Weight  EIP  Weighted  EIP 

Dependent 

Variable 


Response  Tlmea 


Overall  Decision 

.65 

.70 

.71 

.81 

Individual  Acquisition 

— 

— 

.16 

.23 

Time  for  Each  Alternative 

— 

.49 

.48 

.68 

Time  for  Each  Attribute 

— 

.64 

.65 

.75 

Index  of  Subjective  Effort** 

.57 

.56 

.55 

.59 

Errors0 

.68 

.62 

.63 

.64 

aThe  fit  statistics  are  R2  values  for  regressions.  The  sample  sizes  were  733, 
12178,  2963,  and  2313  for  the  overall  decision,  individual  acquisition,  time 
for  each  alternative,  and  time  for  each  attribute  models  respectively. 

**The  fit  statistics  are  R2  values  for  regressions.  The  sample  size  was  737. 
cThe  fit  statistics  are  pseudo-R2  values  for  logistic  regressions.  The  sample 


size  was  840 
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:  7 

iates  of  Time  for  EIP's  (seconds) 


.  of 

EIP 

igation 

READS 

ADDITIONS 

PRODUCTS 

COMPARISONS 

ELIMINATIONS 

DIFFERENCES 

ill  Decision 

1.19* 

.84* 

2.23* 

.09 

1.80* 

.32 

/idual  Acquisition 

-- 

.57* 

1.22* 

.17* 

.01 

.19* 

for  Each 

Ltemative 

1.42* 

.33* 

1.42* 

.16* 

.10 

.28* 

for  Each 
: tribute 

1.22* 

.68* 

1.97* 

.45* 

.26* 

.72* 
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